Uv mapping and compression

ABSTRACT

A machine can be specially configured to generate one or more atlases that include two-dimensional texture maps and their corresponding UV maps from a three-dimensional object, compress the atlases, decompress the atlases, store the atlases, access the atlases, communicate the atlases, apply the texture maps from the atlases to a three-dimensional model, or otherwise process the atlases, the texture maps, the UV maps, or any suitable combination thereof. The atlases, texture maps, UV maps, or any suitable combination thereof can be generated, compiled or otherwise created by the machine in a manner that is computationally efficient to compress and decompress using video compression and decompression techniques.

TECHNICAL FIELD

The subject matter disclosed herein generally relates to the technicalfield of special-purpose machines that facilitate computer graphics,including software-configured computerized variants of suchspecial-purpose machines and improvements to such variants, and to thetechnologies by which such special-purpose machines become improvedcompared to other special-purpose machines that facilitate computergraphics. Specifically, the present disclosure addresses systems andmethods to facilitate “UV” mapping and compression of a UV map.

BACKGROUND

A machine can be configured to generate, compress, decompress, store,communicate, or otherwise process computer graphics that representtwo-dimensional (2D) or three-dimensional (3D) objects. “UV mapping” isthe process of generating a 2D UV map and a 2D texture map from acolored surface in 3D. A texture map is a 2D array of colored pixelsused for texture mapping, which is the process of generating a coloredsurface in 3D from a UV map and a texture map. In this sense, UV mappingand texture mapping can be considers as inverses.

In computer graphics, a UV map refers to an assignment of each 3D pointon a surface to a 2D point in a rectangular domain; that is, a UV map isa mathematical map from the coordinates of a 3D surface to thecoordinates of a 2D image. For example, if the surface is represented by(e.g., expressed as) a list of surface elements (e.g., vertices, voxels,etc.), and the image is represented by a 2D array of pixels, then a UVmap may be represented by a list of indices, the same length as thenumber of surface elements, indicating the index of the pixel in theimage that corresponds to the surface element. UV mapping is the processof generating (e.g., from the list of surface elements) a list ofcorrespondences along with an image (i.e., the texture map) such thatthe color of a surface element is equal to the color of itscorresponding pixel in the texture map.

Mathematically, such an assignment can be represented by a function ƒ:

→

mapping 3D points on a surface

⊂

³ to 2D points in a rectangular domain

∈

². That is, the 2D point (u, v)∈

is assigned to the 3D point (x, y, z)∈

if (u, v)=ƒ(x, y, z). Let I(u, v) denote the color of an image at thepoint (u, v)∈

, and let

(x, y, z) denote the color of the surface at the point (x, y, z)∈

. Here, to be concise, only color is discussed, but more generalattributes besides color, such as transparency, normals, and the like,can be specified.

First, consider the technical problem of determining the color of thesurface C(r, g, b) at each point (x, y, z) in a surface

, given an image {I(u, v):(u, v)∈

} and a UV map {ƒ(x, y, z):(x, y, z)∈

}. This technical problem can be considered as a “forward” technicalproblem, for reasons that will become clear shortly. The technicalsolutions to this technical problem can be expressed in the followingform:

C(x,y,z)=I(ƒ(x,y,z)).

In computer graphics, the process of using this formula to render thecolor of a surface

is called texture mapping, and the image {I(u, v):(u, v)∈

} is called a texture map. The surface

is usually represented by a mesh

=(V,

), where V={(x_(n), y_(n), z_(n))} is a list of vertices and

={(i_(m), j_(m), k_(m))} is a list of faces. Each vertex (x_(n), y_(n),z_(n))∈V is a point in 3D and each face (i_(m), j_(m), k_(m))∈

is a triple of indices into the list V of vertices. For conciseness andclarity, the discussion herein focuses on triangular meshes, but thesystems and methods discussed herein are also applicable to more generalpolygonal meshes, such as quad meshes. The texture map {I(u, v):(u, v)∈

} is usually represented by a 2D array of pixels, and the UV map {ƒ(x,y, z):(x, y, z)∈

} is usually represented by coordinates (u_(n), v_(n)) in one-to-onecorrespondence with the vertices (x_(n), y_(n), z_(n)) of the mesh. Withthese representations, if (x, y, z)∈

is a 3D point on a face of the mesh, then it corresponds to a 2D pointƒ(x, y, z)=(u, v)∈

as follows.

Suppose the face containing (x, y, z) has vertices (x_(n) ₁ , y_(n) ₁ ,z_(n) ₁ ), (x_(n) ₂ , y_(n) ₂ , z_(n) ₂ ), and (x_(n) ₃ , y_(n) ₃ ,z_(n) ₃ ), and suppose that α₁, α₂, α₃ are barycentric coordinates of(x, y, z), namely coefficients 0≤α₁, α₂, α₃≤1 such that

(x,y,z)=α₁(x _(n) ₁ ,y _(n) ₁ ,z _(n) ₁ )+α₂(x _(n) ₂ ,y _(n) ₂ ,z _(n)₂ )+α₃(x _(n) ₃ ,y _(n) ₃ ,z _(n) ₃ ).

Then,

ƒ(x,y,z)=(u,v)=α₁(u _(n) ₁ ,v _(n) ₁ )+α₂(u _(n) ₂ ,v _(n) ₂ )+α₃(u _(n)₃ ,v _(n) ₃ ).

Thus, when a face on the surface is rasterized and rendered, each point(x, y, z) on the face obtains its color by looking up the color in theimage at the corresponding location (u, v). Note that the point (u, v)may not lie exactly at the center of a pixel, so interpolation of somekind may be used to look up the color.

For dynamic (e.g., time-varying) surfaces, typically the mesh vertices(x_(n)(t), y_(n)(t), z_(n)(t)) become functions of time and hence followtrajectories through space and time. Such a mesh can be called a dynamicor animated mesh. Typically, however, in a dynamic mesh, the UVcoordinates (u_(n), v_(n)) remain constant. Thus a single texture mapcan be used for a dynamic mesh.

FIG. 1 is a conceptual diagram illustrating an example of UV mappingapplied to a 3D object; that is, a process for determining a texture map{I(u, v):(u, v)∈

} and a UV map {ƒ(x, y, z):(x, y, z)∈

}, given the color of the surface C(x, y, z) at each point (x, y, z) ina surface

, such that C(x, y, z)=I(ƒ(x, y, z)) holds. This can be considered an“inverse” technical problem relative to the “forward” technical problemdiscussed above. The process for determining such a texture map and a UVmap from a surface can thus be called UV mapping, which is illustratedin FIG. 1.

In UV mapping, for each 3D point (x, y, z) on the surface

, a corresponding 2D point t (u, v) on the texture map is determined,and the color C(x, y, z) (e.g., yellow, red, light blue, or dark green,as shown in FIG. 1) is copied from the surface to the texture map asI(u, v). Since the surface is usually represented by a mesh, thecorrespondence is usually determined by first determining, for each 3Dvertex (x_(n), y_(n), z_(n)) of each face of the mesh, a corresponding2D point (u_(n), v_(n)) on the texture map. Then, the 2D point (u, v)corresponding to a point (x, y, z) on a face is obtained by calculatingthe barycentric coordinates of (x, y, z) with respect to the vertices ofthe face. This correspondence constitutes a UV map.

Once the texture map and the UV map are determined (e.g., by solving the“inverse” problem), they can be used for texture mapping (e.g., to solvethe “forward” problem, such as when texturing the surface of a 3D modelof the original 3D object). The solution to the (“inverse”) UV mappingproblem is often not well-defined. That is, given the colors S on thesurface

, there can be many choices of I and ƒ such that C(x, y, z)=I(ƒ(x, y,z)).

FIG. 2 is a conceptual diagram illustrating examples of charting, whichis one approach to solving the UV mapping problem by defining ƒpiecewise. That is, the domain of ƒ, namely

, is partitioned into non-overlapping pieces

₁,

₂,

₃, . . . such that U_(k)

_(k)=

, and ƒ is defined separately on each piece. The process of determininghow to partition

into pieces

₁,

₂,

₃, . . . is called charting, which is illustrated in FIG. 2. In FIG. 2,boundaries between such non-overlapping pieces are shown in red.

FIG. 3 is a conceptual diagram illustrating an example of UV mappingapplied to a specific charting. Once the pieces of

are determined, the UV map ƒ is determined on each piece

₁,

₂,

₃, . . . . This process is called chart parametrization, and involvesflattening each piece

_(k) onto an image plane in such a way that the flattened pieces ƒ(

_(k)) do not overlap and such that there is little (e.g., minimal) localgeometric distortion between the piece

_(k) and its flattened version ƒ(

_(k)), which is illustrated in FIG. 3, using the same example colorsdescribed above with respect to FIG. 1 and FIG. 2.

The flattened pieces ƒ(

_(k)) are called charts, and the collection of charts is called anatlas, by analogy with a seafarer's maps of the Earth, in whichdifferent pieces of the Earth appear on each chart. Because the surfaceis often represented by a mesh, charting can involve partitioning themesh into sub-meshes. Then, chart parametrization may include assigninga 2D point (u_(n), v_(n)) to each 3D vertex (x_(n), y_(n), z_(n)) ofeach face of each sub-mesh. Then, the atlas becomes a texture map, whichcan be used to re-color the surface. Charting and chart parametrizationare often done jointly, and such a joint process is sometimes calledatlas parametrization. Atlas parametrization is thus a form of UVmapping. As used herein, “atlas” refers both to a texture map of charts,and also to a texture map of charts in combination with the UV map thatunderlies it; that is, atlas refers to a representation of the result oroutput or solution of atlas parametrization or the UV mapping problem.

Atlas parametrization may have a goal of minimizing the distortionresultant from the stretching of the texture. Stretching may beimpossible to avoid when flattening a 3D mesh onto a 2D plane, but itcan be minimized. Texture mapping quality and performance may thereforebenefit from an atlas parametrization algorithm able to minimize suchdistortion. For example, a texture stretch metric can be used tocalculate how much the distortion affects a certain 2D mapping. FIG. 4is a conceptual diagram illustrating an example of how the texture of a3D surface (a) can be stretched by an unrefined atlas parametrization(b) and how the minimization of the texture stretch metric improves thefinal result (c).

For dynamic surfaces, temporal coherence can also be important. Changesin the geometry and topology due to motion can cause overlapping andsharp changes of the 2D polygons. Moreover, if the dynamic mesh does nottrack the surface with sub-pixel accuracy, the colors of the surface candrift in the texture map, causing temporal incoherence of the texturemap. Temporally consistent meshing is a way to maximize temporalcoherence in the texture map.

Compression of texture maps as images is common in computer graphics. Ifthe texture maps are time-varying, as they may be for dynamic surfaces,then compressing the texture maps as video can be done. Compressiontechniques for images and video can be efficient performed usingalgorithms such as Advanced Video Codec (AVC)/H.264 or High EfficiencyVideo Coding (HEVC)/H.265. Maximizing the temporal coherence of thetexture maps is a good way to exploit the compression efficiency of thevideo coding methods, which generally have very high performance whenthey are able to remove the temporal redundancy of a video sequence.

Improvements to the quality of texture map compression may be obtainedby applying tracking techniques. For example, a tracking method maystart by choosing a key-frame, meshing it, and calculating the UV mapfor it. Then, the mesh of the key-frame is fit to the neighboring framesby projecting the texture components, which in the meanwhile have movedin the 3D space, over the same 2D area as the tracked colors of thekey-frame. As a result of this strategy, when the atlas is compressed bya traditional 2D based video encoder, the reduced motion of the scenecan be well exploited, providing better results for the same bit-rates.

FIG. 5 is a conceptual diagram illustrating an example comparison of adecompressed untracked atlas to a decompressed tracked atlas. The toprow shows a decoded sequence representing an untracked atlas. The bottomrow shows a decoded sequence representing a tracked atlas, as well asthe quality improvement attained when the atlas is tracked.

Unfortunately, atlas parametrization is a computationally heavy processwhose complexity increases with the Level of Detail (LOD) of the meshrepresentation of the 3D object. In addition, when the meshes areparametrized on a plane, the textures are affected by stretchingdistortions.

BRIEF DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawings will be provided by the U.S. Patent and Trademark Officeupon request and payment of the necessary fee.

Some embodiments are illustrated by way of example and not limitation inthe figures of the accompanying drawings.

FIG. 1 is a conceptual diagram illustrating an example of UV mappingapplied to a 3D object.

FIG. 2 is a conceptual diagram illustrating examples of charting.

FIG. 3 is a conceptual diagram illustrating an example of UV mappingapplied to a specific charting.

FIG. 4 is a conceptual diagram illustrating an example of how thetexture of a 3D surface can be stretched by an unrefined atlasparametrization and how the minimization of the texture stretch metricimproves the final result.

FIG. 5 is a conceptual diagram illustrating an example comparison of adecompressed untracked atlas to a decompressed tracked atlas.

FIG. 6 is a conceptual diagram illustrating a 3D surface divided intocubic blocks, according to some example embodiments.

FIG. 7 is a conceptual diagram illustrating three orthogonalprojections, according to some example embodiments.

FIG. 8 is a conceptual diagram illustrating occlusion of points,according to some example embodiments.

FIG. 9 is a conceptual diagram illustrating assignment of averaged colorvalues, according to some example embodiments.

FIG. 10 is a conceptual diagram illustrating assignment of copied colorvalues, according to some example embodiments.

FIG. 11 is a conceptual diagram illustrating block-wise occlusion and amost external block, according to some example embodiments.

FIG. 12 is a conceptual diagram illustrating a fully occupied cubicblock and a partially occupied cubic block, according to some exampleembodiments.

FIG. 13 is a conceptual diagram illustrating blocks with a commondominant direction and aligned block centers, according to some exampleembodiments.

FIG. 14 is a conceptual diagram illustrating the filling of adjacentcubic blocks, according to some example embodiments.

FIG. 15 is a conceptual diagram illustrating the filling of non-adjacentcubic blocks, according to some example embodiments.

FIG. 16 is a conceptual diagram illustrating the filling of occludedcubic blocks with averaged color values, according to some exampleembodiments.

FIG. 17 is a conceptual diagram illustrating a sextet of projectedimages, according to some example embodiments.

FIG. 18 is a conceptual diagram illustrating a resized image, accordingto some example embodiments.

FIG. 19 is a conceptual diagram illustrating image partitioning alongthe longer side of an image, according to some example embodiments.

FIG. 20 is a block diagram illustrating operations in the encoding anddecoding of an atlas, according to some example embodiments.

FIG. 21 is a network diagram illustrating a network environment suitablefor generating, storing, compressing, decompressing, communicating, orotherwise processing an atlas or portion thereof (e.g., a UV map),according to some example embodiments.

FIG. 22 is a block diagram illustrating components of a UV map machinesuitable for generating an atlas or portion thereof (e.g., a UV map),according to some example embodiments.

FIG. 23 is a flowchart illustrating operations of a machine inperforming a method of generating an atlas or portion thereof (e.g., aUV map), according to some example embodiments.

FIG. 24 is a block diagram illustrating components of a machine (e.g., adevice) suitable for applying an atlas or portion thereof (e.g., a UVmap) to a 3D model, according to some example embodiments.

FIG. 25 is a flowchart illustrating operations of a machine (e.g., adevice) in performing a method of applying an atlas or portion thereof(e.g., a UV map) to a 3D model, according to some example embodiments.

FIG. 26 is a block diagram illustrating components of a machine (e.g.,device), according to some example embodiments, able to readinstructions from a machine-readable medium and perform any one or moreof the methodologies discussed herein.

DETAILED DESCRIPTION

Example methods (e.g., algorithms) facilitate UV mapping, compression ofa UV map, or both, and example systems (e.g., special-purpose machinesconfigured by special-purpose software) are configured to facilitate UVmapping, compression of a UV map, or both. Examples merely typifypossible variations. Unless explicitly stated otherwise, structures(e.g., structural components, such as modules) are optional and may becombined or subdivided, and operations (e.g., in a procedure, algorithm,or other function) may vary in sequence or be combined or subdivided. Inthe following description, for purposes of explanation, numerousspecific details are set forth to provide a thorough understanding ofvarious example embodiments. It will be evident to one skilled in theart, however, that the present subject matter may be practiced withoutthese specific details.

The systems and methods described herein are specially configured to,inter alia, generate an atlas, including its underlying UV map, by usinga direct orthogonal projection in place of parametrization andaccordingly avoid conventional charting. In various example embodiments,these systems and methods enhance the atlas by performing aquad-tree-based voxel density analysis and grouping, in 2D, colors thatwere originally adjacent in 3D. In addition, the systems and methodsdescribed herein are less affected by the stretching issue, because the2D mapping is performed with a direct orthogonal projection.

In particular, the systems and methods described herein are speciallyconfigured to generate, store, compress, decompress, communicate, orotherwise process a texture map in the form of a special data structure(e.g., a special memory-resident data structure or a special datastream) that represents an atlas (e.g., a texture map and a UV map) andcan be applied independently on any kind of volumetric video (e.g., atime-varying 3D model, potentially having time-varying colors on itssurface). The special data structure can be compressed or decompressedefficiently using a video codec.

Within the special data structure, texture colors are represented byevaluating a block-wise orthogonal projection, where each block is acubic block CB with size N×N×N, and where each orthogonal projection ofthe cubic block is a square block SB with size N×N. The orthogonalprojection of the cubic block is evaluated in the block's dominantdirection, which is automatically chosen by machine from among the threeaxes of three-space (e.g., x, y, and z). The decision about the dominantdirections is made by evaluating a 2D projection for each direction(e.g., normalizing one of the three coordinates) and choosing thedirection that minimizes occlusions (e.g., minimizes occluded area). Analternative implementation is to take into account the inside andoutside of the surface, in which case six possible dominant directionsmay be possible (e.g., +x, −x, +y, −y, +z, −z).

After the evaluation of each orthogonal projection, the 2D N×N blocksare grouped together to avoid excessive fragmentation of the atlas,which may reduce compression efficiency when compressed by a videoencoder. In some example embodiments, a following step, called“filling,” is performed to avoid the presence of edges due to emptyareas that appear when the volumetric content (e.g., the 3D model) doesnot completely fill the N×N area in the dominant direction. In thisfilling operation, a color that corresponds to the average of thetexture information of a cubic block is assigned to all of the emptypixels of the 2D images.

The 2D image is then processed by a video encoder, such as AVC/H.264 orHEVC/H.265, which generates a compressed binary data stream. The binarydata stream can then be stored, communicated, and eventually decoded,such that a decompressed 2D image that contains the original textureinformation is recreated. At the decoder side (e.g., a client device),every projected area is reassigned to a set of 3D coordinates belongingto the volumetric video to recreate the 3D content. This process is thenreproduced, repeating the evaluation of the dominant direction for eachN×N 2D block and grouping the 2D blocks together. In this way, thesystems and methods discussed herein obtain the 3D location of where toassign the color of each 2D pixel.

Following the above steps, it is possible to reduce the amount ofinformation used to represent the texture colors of a volumetric video.In order to simplify the decoding process, an alternative solution canbe implemented to avoid repeating the projection process at the decoderside. In this case, supplemental information (e.g., side information orother metadata) is added to the compressed data stream to indicate thesizes of the mapped 2D areas and their corresponding positions in the 3Dspace. This solution enables a very fast decoding process in exchangefor slightly reduced compression efficiency.

FIG. 6 is a conceptual diagram illustrating a 3D surface divided intocubic blocks, according to some example embodiments. As used herein,“volumetric content” is defined as a set of 3D points

⊂

³. This set of points may informally be referred to as a surface, evenif the set of points is not a 2D manifold embedded in 3D, an alternativenotion of a surface.

The surface

can be represented in any number of ways, such as a mesh, point cloud,and the like. A mesh

=(V,

) includes a set V and a set

, where V={(x_(n), y_(n), z_(n))} is a list of vertices and

={(i_(m), j_(m), k_(m))} is a list of faces. A point cloud is just a setV ={(x_(n), y_(n), z_(n))} of 3D points. The systems and methodsdiscussed herein can be applied to any set of points or surface

⊂

³ independently of its representation.

In accordance with the systems and methods discussed herein, the 3Dsurface

is subdivided into 3D cubic blocks. The colors of the 3D surface

are then represented in the 2D rectangular domain

∈

², which is created by grouping together 2D square blocks. When the typeof block (e.g., 3D or 2D) is readily apparent from context, the word“block” or “blocks” may be used. A cubic block can be defined as the setof points (x, y, z) satisfying the condition:

${CB} = \left\{ {\left. \left( {x,y,z} \right) \middle| {{x_{0} - \frac{N}{2}} < x \leq {x_{0} + \frac{N}{2}}} \right.,{{y_{0} - \frac{N}{2}} < y \leq {y_{0} + \frac{N}{2}}},{{z_{0} - \frac{N}{2}} < z \leq {z_{0} + \frac{N}{2}}}} \right\}$

where (x₀, y₀, z₀) and N are, respectively, the center and the size ofthe edges of the block CB. When the size of the edge of the cubic blocksis fixed, the size of the side square blocks of the 2D image can bedefined as the same.

A square block can be defined as the set of points (u, v) satisfying thecondition:

${SB} = \left\{ {\left. \left( {u,v} \right) \middle| {{u_{0} - \frac{N}{2}} < u \leq {u_{0} + \frac{N}{2}}} \right.,{{v_{0} - \frac{N}{2}} < v \leq {v_{0} + \frac{N}{2}}}} \right\}$

where (u₀, v₀) and N are, respectively, the center and the size of theedges of the block SB.

The aforementioned size of the cubic blocks can also be not fixedinitially. In such a case, the size can be automatically determined by aRate Distortion (RD) optimization algorithm, where the Rate correspondsto the amount of information (e.g., in bits) needed to represent thecompressed colors of the 3D surface, and where the Distortion is thedifference, in terms of quality, between the color of the originalsurface and the reconstructed color obtained after the decompressionprocess.

In accordance with the systems and methods discussed herein, the mappingprocess can be divided into three major steps:

-   -   1. orthogonal projection of the cubic blocks;    -   2. filling of the previously projected blocks; and    -   3. grouping together the projected square blocks (e.g., using a        space partitioning tree technique).

The first major step is orthogonal projection. When the surface

intersects a cubic block, the portion

_(CB) _(k) of the surface

, included in the kth cubic block CB, is defined as the set of pointssatisfying the condition:

$_{CB} = \left\{ {\left. {\left( {x,y,z} \right) \Subset } \middle| {{x_{0} - \frac{N}{2}} < x \leq {x_{0} + \frac{N}{2}}} \right.,{{y_{0} - \frac{N}{2}} < y \leq {y_{0} + \frac{N}{2}}},{{z_{0} - \frac{N}{2}} < z \leq {z_{0} + \frac{N}{2}}}} \right\}$

where (x₀, y₀, z₀) and N are, respectively, the center and size of theedges of the block CB. To simplify the notation hereinafter, the set ofpoints satisfying the above condition can be expressed as

_(CB)={(x, y, z)}.

An orthogonal projection is applied to the portion of surface

_(CB). The surface included in a cubic block is transformed into threedifferent 2D square areas. Each of them includes an orthogonalprojection aligned to a different one of the three axes of a Cartesiancoordinate system: x-axis, y-axis and z-axis.

The orthogonal projection aligned along the x-axis implies that the xcomponent is set equal to 0:

P _(x=0)=(x,y,z)→(0,y,z)

The same condition, applied to the other components, is valid for thetwo orthogonal projections obtained along the y-axis and z-axis:

P _(y=0)=(x,y,z)→(x,0,z)

P _(z=0)=(x,y,z)→(x,y,0)

The application of each orthogonal projection, to convert a cubic blockinto a square block, can thus be defined as:

SB _(x) =P _(x=0)

_(CB)

SB _(y) =P _(y=0)

_(CB)

SB _(z) =P _(z=0)

_(CB)

FIG. 7 is a conceptual diagram illustrating these three orthogonalprojections, according to some example embodiments.

To simplify the application of the three orthogonal projections to

_(CB), the coordinate system of the surface

is first shifted to origin of the cube CB. The center of the cube isthen defined as:

$\left. \left( {x_{0},y_{0},z_{0}} \right)\rightarrow\left( {\frac{N}{2},\frac{N}{2},\frac{N}{2}} \right) \right.$

The shifted portion of surface

_(CB) is then called

_(CB) ^(T) and defined as:

$_{CB}^{(T)} = \left\{ \left( {x,y,z} \right) \middle| {\left\{ {{x + \left( {x_{0} - \frac{N}{2}} \right)},{y + \left( {y_{0} - \frac{N}{2}} \right)},{z + \left( {z_{0} - \frac{N}{2}} \right)}} \right\} \in _{CB}} \right\}$

The corresponding projected square block is then defined as:

SB ^((T))={(u,v)|0<u≤N,0<v≤N}

When projecting along the x-axis, the x component is set equal to 0. Thepoints of

_(CB) ^((T)) projected along the x-axis will then occupy the positionsdefined as:

(u,v)∈SB _(x) ^((T)) =P _(x=0)

_(CB)

In a similar way, projecting along the y-axis and z-axis, the projectedpoints will occupy, respectively, the positions defined as:

(u,v)∈SB _(y) ^((T)) =P _(y=0)

_(CB)

(u,v)∈SB _(z) ^((T)) =P _(z=0)

_(CB)

Once the three orthogonal projections are evaluated, only one isselected to be the mapping, in the 2D space, of the cubic block. Theselected orthogonal projection is the one that corresponds to theprojection aligned to the axis parallel to the dominant direction. Whenprojecting a surface into a 2D space, a certain portion of surface canbe occluded. As used herein, a 3D point in the cubic block is said to be“occluded” if there is another 3D point in the cubic block projecting tothe same 2D point in the square block. The projection onto the squareblock of the 3D points in the cubic block that is occluded is called theoccluded area. FIG. 8 is a conceptual diagram illustrating occlusion ofpoints, according to some example embodiments.

The dominant direction may be determined by various methods in variousexample embodiments of the systems and methods discussed herein. In onemethod, the dominant direction is the one that projects into a squareblock the surface included in a cubic block, minimizing the occludedarea. In another method, the dominant direction is the one most closelyaligned to the surface normal. In another method, the dominant directionis the one most closely aligned to the minor axis of an ellipsoid fit tothe set of points in the cubic block. Various other methods are alsopossible.

In some example embodiments of the systems and methods discussed herein,the dominant direction is determined by minimizing the occluded area.For example, minimizing the occluded area can be performed by maximizingthe projected area. When the projections used are the three previouslydefined, such maximization can be done among the three sets of pointsdefined by SB_(x) ^((T)), SB_(y) ^((T)) and SB_(z) ^((T)).Transformations or other kinds of projections may be used, according tovarious example embodiments. In such cases, the maximization may beperformed over different sets of points. Accordingly, for purposes ofminimizing the occluded area, the dominant direction D may be the onecorresponding to the projection P that maximizes the area of theprojected surface:

SB _(D) ^((T))=argmax{∫_(SB) _((T)) dudv|SB ^((T)) ∈{SB _(x) ^((T)) ,SB_(y) ^((T)) ,SB _(z) ^((T))}}

Once the dominant direction is found, a 2D representation of the colorinformation can be assigned to the 2D square block. According to variousexample embodiments of the systems and methods discussed herein, suchcolor information is obtained by choosing among two options:

-   -   1. the color values of the 2D square block can be assigned by        averaging the color values of the 3D points, of the surface,        aligned along the dominant direction; or    -   2. to reduce complexity, the color values of the 2D square block        can be assigned by copying the color values of the first 3D        points, of the surface, aligned along the dominant direction,        excluding the colors of the occluded 3D points.        FIG. 9 is a conceptual diagram illustrating an example        assignment of averaged color values (the first option), while        FIG. 10 is a conceptual diagram illustrating an example        assignment of copied color values (the second option). Any of        various kinds of interpolation techniques can be used to        represent the color information of the projected surface.

At this stage, the systems and methods discussed herein have obtained a2D projected version of each cubic block of the surface. It can behelpful to consider a generic cubic block CB, with center positionc=(x₀, y₀, z₀) and dominant direction D, and also having a correspondinggeneric square block SB∈

²:

Next, the systems and methods discussed herein identify only thoseprojections that correspond to the cubic blocks respecting the conditionof being the most external layer of the surface

. As used herein, the phrase “most external layer” refers to thoseblocks that, when observed from the dominant direction, are not occludedby any other block. FIG. 11 is a conceptual diagram illustratingblock-wise occlusion and a most external block, according to someexample embodiments.

To define the most external blocks, it can be helpful to use thenotation CB_(ext). To define the blocks belonging to the set CB_(ext),the set B is defined as including all of the cubic blocks, dividedwithin three sets of blocks depending on the dominant direction B_(x),B_(y), B_(z). The conditions for a block to belong to the set CB_(ext)are given by:

{tilde over (B)} _(x) ⊂B _(x) :={tilde over (B)} _(x) ={B _(i) :x_(i)=min_(j) x _(j) ∨x _(i)=max_(j) x _(j)}

{tilde over (B)} _(v) ⊂B _(y) :={tilde over (B)} _(y) ={B _(i) :y_(i)=min_(j) y _(j) ∨y _(i)=max_(j) y _(j)}

{tilde over (B)} _(z) ⊂B _(z) :={tilde over (B)} _(z) ={B _(i) :z_(i)=min_(j) y _(j) ∨z _(i)=max_(j) y _(j)}

All of these sets {tilde over (B)} thus belong to CB_(ext):

B _(ext) ={tilde over (B)} _(x) ∪{tilde over (B)} _(y) ∪{tilde over (B)}_(z)

Each set {tilde over (B)} can be divided into two sets {tilde over (B)}⁻and {tilde over (B)}₊, where:

{tilde over (B)} _(x−) ⊂{tilde over (B)} _(x) :{tilde over (B)} _(x−)={{tilde over (B)} _(i) :x _(i)=min_(j) x _(j) }; {tilde over (B)} _(x+)⊂{tilde over (B)} _(x) :{tilde over (B)} _(x+) ={{tilde over (B)} _(i):x _(i)=max_(j) x _(j)}

{tilde over (B)} _(y−) ⊂{tilde over (B)} _(y) :{tilde over (B)} _(y−)={{tilde over (B)} _(i) :y _(i)=min_(j) y _(j) }; {tilde over (B)} _(y+)⊂{tilde over (B)} _(y) :{tilde over (B)} _(y+) ={{tilde over (B)} _(i):y _(i)=max_(j) y _(j)}

{tilde over (B)} _(z−) ⊂{tilde over (B)} _(z) :{tilde over (B)} _(z−)={{tilde over (B)} _(i) :z _(i)=min_(j) z _(j) }; {tilde over (B)} _(z+)⊂{tilde over (B)} _(z) :{tilde over (B)} _(z+) ={{tilde over (B)} _(i):z _(i)=max_(j) x _(j)}

The 2D projections SB corresponding to the cubic blocks, belonging toeach of the 6 sets explained above, are grouped together into 6 sets ofsquare blocks grouped separately.

G _(x−) ,G _(x+) ,G _(y−) ,G _(y+) ,G _(z−) ,G _(z+)

The generic set is accordingly called G.

The second major step of the mapping process is filling the previouslyprojected blocks. The previous projection produced 6 sets of squareblocks, which are 2D sparse matrices. The sparse nature of each block SBis due to the fact that a cubic block CB may not be fully occupied whenprojected along its dominant direction. FIG. 12 is a conceptual diagramillustrating a fully occupied cubic block and a partially occupied cubicblock, according to some example embodiments. The presence of the emptyspaces will likely cause a reduction of compression efficiency when theUV map is compressed by a video encoder. Therefore, in order to reducethe impact of the empty areas, a step called filling is performed.

The filling can be performed by grouping together the 2D projections,each already grouped into one of the sets G, with the projections ofother cubic blocks belonging to B_(int). For a generic blockCB_(ext)=B_(ext), the considered blocks of the set CB_(int)⊂B_(int) arethe ones which meet the following two conditions:

-   -   1. If the block CB_(int) is adjacent to the block CB_(ext),        there exists a point of intersection between the surface        and the boundary between CB_(ext) and CB_(int). Otherwise, if        the block CB_(int) is not adjacent to the block CB_(ext), there        exists a point of intersection between the surface        and the boundary between CB_(int) and the blocks, belonging to        B_(int), that fulfill the previous statement.    -   2. With respect to the dominant direction D of the block        CB_(ext), the values of the coordinates (x₀, y₀, z₀) of the        center of CB_(int) that are different than the value        corresponding to D, are equal to each other. For example, if D        corresponds to the projection P_(x=0), the values used are the        ones for y₀, z₀. In such a case, in order to verify the        condition, this condition becomes:

y _(ext,0) =y _(int,0) ∧z _(ext,0) =z _(int,0)

FIG. 13 is a conceptual diagram illustrating blocks with a commondominant direction and aligned block centers, according to some exampleembodiments.

In FIG. 13, the uppermost diagram illustrates adjacent two blocks,denoted “A” and “B,” which have a common boundary that intersects thesurface

. In the middle diagram within FIG. 13, the two blocks denoted “A” and“B” are not adjacent, but there is an intersection between the surface

and a common boundary between block “B” and another (e.g., intervening)block that fulfills the same condition shown in the uppermost diagram inFIG. 13. The uppermost diagram and the middle diagram in FIG. 13therefore illustrate examples of the first condition discussed above. Incontrast, the bottommost diagram within FIG. 13 illustrates an exampleof the second condition discussed above. In the bottommost diagram ofFIG. 13, X is the dominant direction; Y_ext is equal to Y_int; and Z_extis equal to Z_int.

The filling operation can be implemented in at least the following twodifferent ways, depending on whether the pixels of the 2D square block,projected in the orthogonal projection operation, were already occupied(e.g., indicating presence of occlusions) or not (e.g., corresponding toan empty area of the projected cubic block).

When one of the most external cubic blocks is projected to a 2D squareblock, there may be an empty area along the dominant direction whichremains empty in the square block. When the 2D image, containing thesquare blocks, is compressed, empty areas will likely produce highfrequency content which is not handled by the video encoder as well asthe low frequency content. The filling step reduces the amount of highfrequency content by filling these empty areas.

In some situations, cubic blocks are adjacent. Suppose that both thegeneric block CB_(ext) and the adjacent block CB_(int) meet the twoaforementioned conditions. The portion of surface

included in CB_(int) is not fully occluded by the portion of

, included in CB_(int), along the dominant direction D. In this case,the two portions of the surface, included in CB_(ext) and CB_(int), areprojected together along the dominant direction D, to the same 2D squareblock SB. FIG. 14 is a conceptual diagram illustrating the filling ofadjacent cubic blocks, according to some example embodiments.

In other situations, cubic blocks are non-adjacent. Suppose that boththe generic block CB_(ext) and the adjacent block CB_(int) meet thesecond of the two aforementioned conditions. The portion of surface

included in CB_(int) is not fully occluded by the portion of

included in CB_(ext) along the dominant direction D. The cubic blockcalled CB_(int) does not meet the either of the two aforementionedconditions with any other block, different from CB_(ext), along theother two directions different than D. In this case, the two portions ofthe surface that are included in CB_(ext) and CB_(int) are projectedtogether along the dominant direction D to the same 2D square block SB.FIG. 15 is a conceptual diagram illustrating the filling of non-adjacentcubic blocks, according to some example embodiments.

Occlusion may be present in certain situations (e.g., involving adjacentcubic blocks). Suppose that both the generic block CB_(ext) and theadjacent block CB_(int) meet the two aforementioned conditions, but theportion of surface

included in CB_(int) is fully or partially occluded by the portion of Sthat is included in CB_(ext) along the dominant direction D. In thiscase, the two portions of the surface that are included in CB_(ext) andCB_(int) are projected together along the dominant direction D to thesame 2D square block SB. The color values of SB are assigned byaveraging the color values of the 3D surface aligned along the dominantdirection D. FIG. 16 is a conceptual diagram illustrating the filling ofoccluded cubic blocks with such average color values, according to someexample embodiments.

At this stage, the systems and methods discussed herein have obtained 6sets of square blocks, including the color information of the mostexternal projected blocks B_(ext), plus performed the above-describedfilling step based on the color information of some of the non-externalblocks that meet the aforementioned conditions.

The portion of the 3D surface not projected so far is stored andprocessed again in a loop. For each loop iteration, 6 different sets ofsquare blocks are generated in accordance with the systems and methodsdiscussed herein. The loop ends when a certain percentage of the 3Dsurface is projected. The remaining part of the surface may then behandled independently, after the third major step in the mappingprocess, which involves grouping together the projected square blocks(e.g., using a space partitioning tree). Before the third major step,the sets are reorganized into 2D matrices. To facilitate thisreorganization, it can be helpful to shift from the continuous space tothe discrete space:

→

.

Recalling the generic set G discussed above, the block SB is a 2D sparsematrix where the positions of the non-zero values are defined by thelist of coordinates (u, v), as previously defined. The values (u, v) arevalid when the surface

is shifted to the center of the cube CB. When shifting the system backas the original surface S, the list of coordinates of SB can be definedas (u′, v′).

Among the set G, the values (u′, v′) of each square block SB are unique.In view of this uniqueness, the coordinates of all blocks in the set canbe grouped together:

g=(u′,v′)

To each couple of values of the list g there will be a vector, with size[1, 3], containing the color information of the surface

at the position (x, y, z) projected to each 2D point (e.g., byprojecting, filling, averaging, or any suitable combination thereof, asdescribed above). For commodity, the list of colors that correspond to gis called:

C=(r,g,b)

The color vector at a generic position c of the list C corresponds tothe couple of coordinates g(c).

The uniqueness of each square block enables reorganization of eachsquare block into a matrix M. In the matrix, the positions of the valuesare defined by the color list corresponding to g. The matrix M hasdimensions (I, J) that equal the maximum values of the two coordinatesdifferent from the ones corresponding to the dominant direction D.

Accordingly, matrices M_(x−) and M_(x+) have dimensions (I, J)=(y_(max),z_(max)). Following a similar process, the dimensions of matrices M_(y−)and M_(y+) are (x_(max), z_(max)), and the dimensions of matrices M_(z−)and M_(z+) are (x_(max), y_(max)).

The values included in each matrix are 0 when no 3D point is projected,and a vector that includes the color of the projected surface point,when there is a projected 3D point. This may be expressed as:

${M = \begin{bmatrix}v_{1,1} & \ldots & v_{0,J} \\\vdots & \ddots & \vdots \\v_{I,0} & \ldots & v_{I,J}\end{bmatrix}};{v_{i,j} = \left\{ \begin{matrix}{{{{C(c)}\mspace{14mu} {if}\mspace{14mu} \left( {i,j} \right)} \in {g\mspace{14mu} \Lambda \mspace{14mu} {g(c)}}} = \left( {i,j} \right)} \\{0\mspace{14mu} {otherwise}}\end{matrix} \right.}$

The third major step in the mapping process is grouping together theprojected square blocks (e.g., using a space partitioning tree). Thismay be performed by processing the six matrices generated in eachiteration of the projection and filling stages, in order to generate theUV map. In particular, the six matrices may be processed using a spacepartitioning tree.

FIG. 17 is a conceptual diagram illustrating a sextet of projectedimages, according to some example embodiments. A space partitioning treecan be applied independently on each matrix previously created. As athreshold matter, each matrix may be initially resized. This may beperformed for either or both of at least two reasons:

-   -   1. to facilitate the space partition tree, the sizes should be        equal to 2^(n), n=1, 2 . . . N; or    -   2. to minimize the 2D areas corresponding to empty 3D space.        For each matrix, the new size may be the minimum size that still        includes all of the non-zero values of the matrix.

Accordingly, the minimum indices pointing to a non-zero value can beexpressed as:

i _(min)=min_(i)(v _(i,j)≠0)

j _(min)=min_(j)(v _(i,j)≠0)

The resized matrix {circumflex over (M)} thus has the size:

Î=(I−i _(min))+[2^(n) ^(i) −((I−i _(min)))]

Ĵ=(J−j _(min))+[2^(n) ^(j) −((J−j _(min)))]

where:

n _(i)=min_(n)2^(n)|2^(n)>(I−i _(min))

n _(j)=min_(n)2^(n)|2^(n)>(J−j _(min))

The new matrix can be denoted as {circumflex over (M)}.

FIG. 18 is a conceptual diagram illustrating a resized image, accordingto some example embodiments. The resized image is processed by apartitioning tree (e.g., a space partitioning tree) that is configuredto divide the area into sub-images that each include an amount ofprojected surface that is higher than a defined occupancy thresholdTh_(occ).

In applying the partitioning tree, as a first internal step, the systemsand methods discussed herein initially calculate the percentage ofprojected surface in the whole matrix. This percentage corresponds tothe ratio between the number of elements of the resized matrix, withsize (

,

), and the number of non-zero elements therein, which is also thecardinality of the vector C.

${Occ} = \frac{\times}{\overset{\_}{\overset{\_}{C}}}$

If c≥Th_(occ), the application of the partitioning tree stops;otherwise, it follows to the second internal step. The image is dividedinto two sub-images along the longer side of the image, such that thetwo sub-images will have equal areas and dimensions (

,

).

( , ) = { ( 2 , ) if   I 1 ≥ J 1 ( , 2 ) if   I 1 < J 1

FIG. 19 is a conceptual diagram illustrating image partitioning alongthe longer side of an image, according to some example embodiments.

After the first division, each of two sub-images is analyzedindependently, checking again to determine whether the amount ofprojected surface is higher than the threshold. If the result isnegative, the sub-image is once again subdivided but, this time, alongthe opposite direction.

( , ) = { ( I ^ 2 , 2 ) if   I 1 ≥ J 1 ( 2 , , ) if   I 1 < J 1

The sizes of the partitioning at the generic stage may be predefined andmay depend on whether the stage number is odd or even.

$\left( {{\hat{I}}_{2\; i},{\hat{J}}_{2\; i}} \right) = \left\{ {{\begin{matrix}\left( {\frac{{\hat{I}}_{{2i} - 1}}{2},{\hat{J}}_{{2i} - 1}} \right) & {{{if}\mspace{14mu} I_{{2i} - 1}} \geq J_{{2i} - 1}} \\\left( {{\hat{I}}_{{2i} - 1},\frac{{\hat{J}}_{{2i} - 1}}{2}} \right) & {{{if}\mspace{14mu} I_{{2i} - 1}} < J_{{2i} - 1}}\end{matrix}\left( {{\hat{I}}_{{2i} + 1},{\hat{J}}_{{2i} + 1}} \right)} = \left\{ \begin{matrix}\left( {{\hat{I}}_{2i},\frac{{\hat{J}}_{{2i} - 1}}{2}} \right) & {{{if}\mspace{14mu} I_{{2i} - 1}} \geq J_{{2i} - 1}} \\\left( {\frac{{\hat{I}}_{{2i} - 1}}{2},{\hat{J}}_{2i}} \right) & {{{if}\mspace{14mu} I_{{2i} - 1}} < J_{{2i} - 1}}\end{matrix} \right.} \right.$

This process may be reiterated until one of the following 2 conditionsholds true:

-   -   1. the 3D surface projected to the 2D image occupies more than a        certain percentage (e.g., threshold percentage) of the 2D image;        or    -   2. the partitioned area has dimensions equal or less than        predefined minimum dimensions (e.g., 8×8 pixels).        When one of the two aforementioned conditions is verified as        being true, the sub-image is then stored in the final UV map.        The final UV map is a special 2D image that includes all of the        sub-images produced by the third major step in the mapping        process (e.g., the partitioning tree stage).

According to various example embodiments of the systems and methodsdiscussed herein, the position, in UV coordinates, of each sub-image inthe UV map is determined by performing an empty areas minimization step.The goal of the empty areas minimization step is to minimize the amountof mapped atlas corresponding to empty 3D space. When the 2D atlasincludes an empty space, a 2D image encoder will likely spend anundesired amount of bits to represent such empty space without a realbenefit to the reconstruction of the 3D surface at the decoder side.Thus, by minimizing the empty areas, the amount of undesired bit usageis correspondingly minimized.

The empty area minimization step starts by checking the previouslymapped 2D atlas. If there is no other sub-image already mapped, thecurrent, and first, sub-image is mapped in the top left corner of theimage. Otherwise, the presence of empty space is calculated.

The empty space, in the UV matrix, contains zero values. These areas canbe represented as Z. The top left corner of each Z is described in thecoordinates of the matrix UV as (u_(Z), v_(Z)), and the correspondingsize of each Z is defined as(I_({tilde over (Z)}),J_({tilde over (Z)})).

If there is a 2D area corresponding to empty space, with equal or higherdimensions compared to those of the current sub-image, the top leftcorner of the sub-image is assigned to the top left corner of the foundempty area. If no empty area is found with dimensions greater than orequal to the ones of the sub-image, the 2D atlas is re-sized to fit theinformation to be mapped.

For the generic sub-matrix {tilde over (M)} with size (Ĩ,{tilde over(J)}), the UV mapping to the UV matrix with size (u_(UV), v_(UV)) can bedescribed as:

UV(u _({tilde over (Z)}) :u _({tilde over (Z)}) +I,v _({tilde over (Z)}):v _({tilde over (Z)}) +{tilde over (J)}) if ∃{{tilde over (Z)}⊂Z:I_({tilde over (Z)}) >Ĩ∧J _({tilde over (Z)}) >{tilde over (J)}}

UV(u _(UV) :u _(uv) +Ĩ,0:{tilde over (J)}) otherwise

In the latter situation, the matrix UV is then resized to fit the newinformation.

This space partitioning tree technique is performed until a certainpercentage (e.g., threshold percentage) of the surface

is projected. To include, in the UV matrix, the remaining surface, whichwas already subdivided into cubic blocks, the projection of each CB ishandled separately, according to some example embodiments. For example,all of the blocks with size N×N×N may be projected into square blocks SBwith size N×N, in accordance with the above-specified equations, where:

{tilde over (M)}=SB

(Ĩ,{tilde over (J)})=(N,N)

Once the atlas, or the atlases in case of dynamic content, are created,the 2D images are compressed by a video-capable encoder that isconfigured to produce a compressed data stream (e.g., compressedbit-stream) that, when received and decoded by a compatible decoder,enables the decoder to reconstruct a decoded version of the 2D atlas.FIG. 20 is a block diagram illustrating operations in the encoding anddecoding of an atlas, according to some example embodiments.

FIG. 20 also shows data flows within an end-to-end system that includesan encoding device (e.g., an encoder) and a decoding device (e.g., adecoder), according to some example embodiments. In solving thetechnical problem of compression of colored geometry, an end-to-endsystem may include an encoder and a decoder. The encoder encodes boththe geometry data and the color data into a data stream (e.g., a bitstream), and the decoder decodes the encoded geometry data and theencoded color data from the data stream. In some example embodiments,the data stream may be segmented into a geometry bit stream and a colorbit stream.

In accordance with certain example embodiments of the systems andmethods discussed herein, the methodologies described herein for UVmapping are used for both encoding and decoding colored geometry. Inparticular, at both the encoder and decoder, certain example embodimentsof the systems and methods discussed herein generate a UV map from thegeometry data only. That is, in such example embodiments, no colorinformation is used to generate the UV map. Therefore, the UV map can begenerated identically at both the encoder and the decoder withoutsending any information that describes the UV map from the encoder tothe decoder. This approach contrasts with other technical solutions forcompression of colored geometry that, despite performing UV mapping inways other than those described herein, nonetheless send at least someinformation from the encoder to the decoder to describe the UV map. Thesystems and methods described herein can avoid sending this additionaldata and are therefore improved over alternative systems and methods byvirtue of at least this feature. Thus, according to certain exampleembodiments of the systems and methods described herein, UV mapping candepend only on geometry data, and such example embodiments can avoidtransmitting any parameters of the UV mapping from the encoder to thedecoder.

At the decoding side (e.g., decoding device), after the decoded atlas isreconstructed, its colors are assigned to the corresponding 3D surface.The inverse projection of the colors is performed by evaluating theabove-described three major steps of the atlas mapping, namely:

-   -   1. orthogonal projection of the cubic blocks;    -   2. filling of the previously projected blocks; and    -   3. grouping together the projected square blocks (e.g., using a        space partitioning tree).

By repeating this process, the systems and methods described hereindetermine exactly the size of each sub-image (e.g., as evaluated by thepartitioning tree technique) and its position in the UV atlas.

The colors are then assigned to the corresponding area of the 3D surfaceby projecting them according the dominant directions calculated at thedecoder side.

If there are occluded areas of the 3D surface, the color information canbe assigned in at least the following two example ways:

-   -   1. by replicating the same color of the first projected pixel        along the dominant direction; or    -   2. by color interpolation between or among the closest available        areas of the surface.        The latter approach provides better quality but requires        additional computational resources compared to the former        approach.

In some example embodiments of the systems and methods discussed herein,decoding proceeds according to an alternative decoding process that doesnot require the evaluation of the atlas mapping at the decoder side.Instead, the size and the positions, in both UV coordinates and 3Dcoordinates of the surface, of each sub-image are transmitted assupplemental information (e.g., side information or other metadata)together with the compressed colors. This alternative method uses anadditional amount of data (e.g., additional bits) in the compressed datastream but reduces the computational load on the decoder. Furthermore,to avoid drift between the encoding and the decoding process, when thegeometry information of the 3D surface is also compressed, the atlasmapping may be performed on the decoded geometry information.

FIG. 21 is a network diagram illustrating a network environment 2100suitable for generating, storing, compressing, decompressing,communicating, or otherwise processing an atlas or portion thereof(e.g., a UV map), according to some example embodiments. The networkenvironment 2100 includes a UV map machine 2110, a database 2115, anddevices 2130 and 2150, all communicatively coupled to each other via anetwork 2190. The UV map machine 2110, with or without the database2115, may form all or part of a cloud 2118 (e.g., a geographicallydistributed set of multiple machines configured to function as a singleserver), which may form all or part of a network-based system 2105(e.g., a cloud-based server system configured to provide one or morenetwork-based services to the devices 2130 and 2150). The UV map machine2110 and the devices 2130 and 2150 may each be implemented in aspecial-purpose (e.g., specialized) computer system, in whole or inpart, as described below with respect to FIG. 26.

Also shown in FIG. 21 are users 2132 and 2152. One or both of the users2132 and 2152 may be a human user (e.g., a human being), a machine user(e.g., a computer configured by a software program to interact with thedevice 2130 or 2150), or any suitable combination thereof (e.g., a humanassisted by a machine or a machine supervised by a human). The user 2132is associated with the device 2130 and may be a user of the device 2130.For example, the device 2130 may be a desktop computer, a vehiclecomputer, a tablet computer, a navigational device, a portable mediadevice, a smart phone, or a wearable device (e.g., a smart watch, smartglasses, smart clothing, or smart jewelry) belonging to the user 2132.Likewise, the user 2152 is associated with the device 2150 and may be auser of the device 2150. As an example, the device 2150 may be a desktopcomputer, a vehicle computer, a tablet computer, a navigational device,a portable media device, a smart phone, or a wearable device (e.g., asmart watch, smart glasses, smart clothing, or smart jewelry) belongingto the user 2152.

Any of the systems or machines (e.g., databases and devices) shown inFIG. 21 may be, include, or otherwise be implemented in aspecial-purpose (e.g., specialized or otherwise non-conventional andnon-generic) computer that has been modified to perform one or more ofthe functions described herein for that system or machine (e.g.,configured or programmed by special-purpose software, such as one ormore software modules of a special-purpose application, operatingsystem, firmware, middleware, or other software program). For example, aspecial-purpose computer system able to implement any one or more of themethodologies described herein is discussed below with respect to FIG.26, and such a special-purpose computer may accordingly be a means forperforming any one or more of the methodologies discussed herein. Withinthe technical field of such special-purpose computers, a special-purposecomputer that has been specially modified (e.g., configured byspecial-purpose software) by the structures discussed herein to performthe functions discussed herein is technically improved compared to otherspecial-purpose computers that lack the structures discussed herein orare otherwise unable to perform the functions discussed herein.Accordingly, a special-purpose machine configured according to thesystems and methods discussed herein provides an improvement to thetechnology of similar special-purpose machines.

As used herein, a “database” is a data storage resource and may storedata structured as a text file, a table, a spreadsheet, a relationaldatabase (e.g., an object-relational database), a triple store, ahierarchical data store, or any suitable combination thereof. Moreover,any two or more of the systems or machines illustrated in FIG. 21 may becombined into a single system or machine, and the functions describedherein for any single system or machine may be subdivided among multiplesystems or machines.

The network 2190 may be any network that enables communication betweenor among systems, machines, databases, and devices (e.g., between the UVmap machine 2110 and the device 2130). Accordingly, the network 2190 maybe a wired network, a wireless network (e.g., a mobile or cellularnetwork), or any suitable combination thereof. The network 2190 mayinclude one or more portions that constitute a private network, a publicnetwork (e.g., the Internet), or any suitable combination thereof.Accordingly, the network 2190 may include one or more portions thatincorporate a local area network (LAN), a wide area network (WAN), theInternet, a mobile telephone network (e.g., a cellular network), a wiredtelephone network (e.g., a plain old telephone system (POTS) network), awireless data network (e.g., a WiFi network or WiMax network), or anysuitable combination thereof. Any one or more portions of the network2190 may communicate information via a transmission medium. As usedherein, “transmission medium” refers to any intangible (e.g.,transitory) medium that is capable of communicating (e.g., transmitting)instructions for execution by a machine (e.g., by one or more processorsof such a machine), and includes digital or analog communication signalsor other intangible media to facilitate communication of such software.

FIG. 22 is a block diagram illustrating components of the UV map machine2110, according to some example embodiments. The UV map machine 2110 isshown as including a 3D object accessor 2210, an atlas generator 2220,and an atlas provider 2230, all configured to communicate with eachother (e.g., via a bus, shared memory, or a switch).

As shown in FIG. 22, the 3D object accessor 2210, the atlas generator2220, the atlas provider 2230, or any suitable combination thereof, mayform all or part of an app 2200 (e.g., a server-side application, aclient-side application, a mobile app, or any suitable combinationthereof) that is stored (e.g., installed) on the UV map machine 2110(e.g., responsive to or otherwise as a result of data being received viathe network 2190). Furthermore, one or more processors 2299 (e.g.,hardware processors, digital processors, or any suitable combinationthereof) may be included (e.g., temporarily or permanently) in the app2200, the 3D object accessor 2210, the atlas generator 2220, the atlasprovider 2230, or any suitable combination thereof.

Any one or more of the components (e.g., modules) described herein maybe implemented using hardware alone (e.g., one or more of the processors2299) or a combination of hardware and software. For example, anycomponent described herein may physically include an arrangement of oneor more of the processors 2299 (e.g., a subset of or among theprocessors 2299) configured to perform the operations described hereinfor that component. As another example, any component described hereinmay include software, hardware, or both, that configure an arrangementof one or more of the processors 2299 to perform the operationsdescribed herein for that component. Accordingly, different componentsdescribed herein may include and configure different arrangements of theprocessors 2299 at different points in time or a single arrangement ofthe processors 2299 at different points in time. Each component (e.g.,module) described herein is an example of a means for performing theoperations described herein for that component. Moreover, any two ormore components described herein may be combined into a singlecomponent, and the functions described herein for a single component maybe subdivided among multiple components. Furthermore, according tovarious example embodiments, components described herein as beingimplemented within a single system or machine (e.g., a single device)may be distributed across multiple systems or machines (e.g., multipledevices).

FIG. 23 is a flowchart illustrating operations of a machine (e.g., UVmap machine 2110) in performing a method 2300 of generating an atlas orportion thereof (e.g., a UV map), according to some example embodiments.Operations in the method 2300 may be performed by the UV map machine2110, the device 2130, or any suitable combination thereof, usingcomponents (e.g., modules) described above with respect to FIG. 22,using one or more processors 2299 (e.g., microprocessors or otherhardware processors), or using any suitable combination thereof. Asshown in FIG. 23, the method 2300 includes operations 2310, 2320, and2330.

In operation 2310, the 3D object accessor 2210 accesses a 3Drepresentation of the 3D object in accordance with any one or more ofthe methodologies discussed herein (e.g., by accessing a database thatstores such a representation in the form of a mesh, voxels, or anysuitable combination thereof). Accordingly, the 3D object accessor 2210accesses a 3D representation of a 3D object, where the 3D representationdefines a 3D surface of the 3D object.

In operation 2320, the atlas generator 2220 generates an atlas accordingto any one or more of the methodologies described herein. Accordingly,the atlas generator 2220 may generate an atlas of color informationbased on the 3D representation accessed in operation 2310. For example,this may be performed by calculating orthogonal projections of cubicblocks and grouping at least some of the resulting square blocks. Insome example embodiments, dominant directions of at least some of thecubic blocks are determined (e.g., to determine which face of a boundingvolume is to be used for the orthogonal projection of the correspondingcubic block). Accordingly, for each cubic block that intersects the 3Dsurface among a plurality of cubic blocks included in a bounding volumethat encompasses the 3D surface, the atlas generator 2220 may calculatean orthogonal projection of the cubic block onto a corresponding 2D faceof the bounding volume, where the orthogonal projection defines acorresponding square block for the cubic block, and then group (e.g.,cluster by moving) at least some of the square blocks that correspond tothe orthogonally projected cubic blocks within a 2D image that specifiesthe color information generated based on the 3D representation of the 3Dobject.

In operation 2330, the atlas provider 2230 provides the generated atlas.For example, the atlas provider 2230 may provide the generated atlas toa device (e.g., device 2130) for processing (e.g., full or partialrendering), a downstream component (e.g., a video compressor or othervideo encoder) of the machine (e.g., UV map machine 2110) that generatedthe atlas, a database (e.g., database 2115) for storage, or any suitablecombination thereof. Accordingly, the atlas provider 2230 provides(e.g., communicates) the generated atlas of the color information, wherethe generated atlas includes the grouped square blocks.

FIG. 24 is a block diagram illustrating components of a machine (e.g.,device 2130) suitable for applying an atlas or portion thereof (e.g., aUV map) generated in accordance with any one or more of themethodologies described herein to a 3D model, according to some exampleembodiments. In FIG. 24, the device 2130 is shown as including an atlasaccessor 2410, a direction determiner 2420, and a texture applicator2430, all configured to communicate with each other (e.g., via a bus,shared memory, or a switch).

As shown in FIG. 24, the atlas accessor 2410, the direction determiner2420, the texture applicator 2430, or any suitable combination thereof,may form all or part of an app 2400 (e.g., a server-side application, aclient-side application, a mobile app, or any suitable combinationthereof) that is stored (e.g., installed) on the device 2130 (e.g.,responsive to or otherwise as a result of data being received via thenetwork 2190). Furthermore, one or more processors 2499 (e.g., hardwareprocessors, digital processors, or any suitable combination thereof) maybe included (e.g., temporarily or permanently) in the app 2400, theatlas accessor 2410, the direction determiner 2420, the textureapplicator 2430, or any suitable combination thereof.

As noted above, any one or more of the components (e.g., modules)described herein may be implemented using hardware alone (e.g., one ormore of the processors 2499) or a combination of hardware and software.For example, any component described herein may physically include anarrangement of one or more of the processors 2499 (e.g., a subset of oramong the processors 2499) configured to perform the operationsdescribed herein for that component. As another example, any componentdescribed herein may include software, hardware, or both, that configurean arrangement of one or more of the processors 2499 to perform theoperations described herein for that component. Accordingly, differentcomponents described herein may include and configure differentarrangements of the processors 2499 at different points in time or asingle arrangement of the processors 2499 at different points in time.Each component (e.g., module) described herein is an example of a meansfor performing the operations described herein for that component.Moreover, any two or more components described herein may be combinedinto a single component, and the functions described herein for a singlecomponent may be subdivided among multiple components. Furthermore,according to various example embodiments, components described herein asbeing implemented within a single system or machine (e.g., a singledevice) may be distributed across multiple systems or machines (e.g.,multiple devices).

FIG. 25 is a flowchart illustrating operations of a machine (e.g.,device 2130) in performing a method 2500 of applying an atlas or portionthereof (e.g., a UV map) to a 3D model, according to some exampleembodiments. Operations in the method 2500 may be performed by the UVmap machine 2110, the device 2130, or any suitable combination thereof,using components (e.g., modules) described above with respect to FIG.24, using one or more processors 2499 (e.g., microprocessors or otherhardware processors), or using any suitable combination thereof. Asshown in FIG. 25, the method 2500 includes operations 2510, 2520, and2530.

In operation 2510, the atlas accessor 2410 accesses an atlas of colorinformation (e.g., from the database 2115, the UV map machine 2110, thedevice 2150, or another suitable source). For example, the atlasaccessor 2410 may receive, decompress, or otherwise access an atlasgenerated in accordance with the discussion above regarding FIG. 23, andthe accessed atlas may have been generated by any one or more of themethodologies described herein. Accordingly, the atlas accessor 2410 mayaccess an atlas of color information generated based on a 3Drepresentation of a 3D object. The atlas may include grouped squareblocks that represent orthogonal projections of cubic blocks included ina bounding volume that encompasses the 3D surface, and the representedcubic blocks may intersect a 3D surface of the 3D object.

In operation 2520, for each square block among the grouped squareblocks, the direction determiner 2420 determines a corresponding 2D faceof the bounding volume by determining a corresponding dominant directionof the corresponding cubic block that intersects a corresponding portionof the 3D surface.

In operation 2530, for each square block among the grouped squareblocks, the texture applicator 2430 assigns corresponding colorinformation to a 3D model of the corresponding portion of the 3D surfaceintersected by the corresponding cubic block.

According to various example embodiments, one or more of themethodologies described herein may facilitate generation, compression,decompression, storage, communication, or other processing of one ormore atlases of color information generated in accordance with themethodologies described herein. Hence, one or more of the methodologiesdescribed herein may facilitate generation, compression, decompression,storage, communication, rendering, or other processing of computergraphics, including texture mapped 3D models of 3D objects, compared tocapabilities of pre-existing systems and methods.

When these effects are considered in aggregate, one or more of themethodologies described herein may obviate a need for certain efforts orresources that otherwise would be involved in working with computergraphics, including atlases, UV maps, texture maps, and any suitablecombination thereof. Efforts expended by a user in this regard may bereduced by use of (e.g., reliance upon) a special-purpose machine thatimplements one or more of the methodologies described herein. Computingresources used by one or more systems or machines (e.g., within thenetwork environment 2100) may similarly be reduced (e.g., compared tosystems or machines that lack the structures discussed herein or areotherwise unable to perform the functions discussed herein). Examples ofsuch computing resources include processor cycles, network traffic,computational capacity, main memory usage, graphics rendering capacity,graphics memory usage, data storage capacity, power consumption, andcooling capacity.

FIG. 26 is a block diagram illustrating components of a machine 2600,according to some example embodiments, able to read instructions 2624from a machine-readable medium 2622 (e.g., a non-transitorymachine-readable medium, a machine-readable storage medium, acomputer-readable storage medium, or any suitable combination thereof)and perform any one or more of the methodologies discussed herein, inwhole or in part. Specifically, FIG. 26 shows the machine 2600 in theexample form of a computer system (e.g., a computer) within which theinstructions 2624 (e.g., software, a program, an application, an applet,an app, or other executable code) for causing the machine 2600 toperform any one or more of the methodologies discussed herein may beexecuted, in whole or in part.

In alternative embodiments, the machine 2600 operates as a standalonedevice or may be communicatively coupled (e.g., networked) to othermachines. In a networked deployment, the machine 2600 may operate in thecapacity of a server machine or a client machine in a server-clientnetwork environment, or as a peer machine in a distributed (e.g.,peer-to-peer) network environment. The machine 2600 may be a servercomputer, a client computer, a personal computer (PC), a tabletcomputer, a laptop computer, a netbook, a cellular telephone, a smartphone, a set-top box (STB), a personal digital assistant (PDA), a webappliance, a network router, a network switch, a network bridge, or anymachine capable of executing the instructions 2624, sequentially orotherwise, that specify actions to be taken by that machine. Further,while only a single machine is illustrated, the term “machine” shallalso be taken to include any collection of machines that individually orjointly execute the instructions 2624 to perform all or part of any oneor more of the methodologies discussed herein.

The machine 2600 includes a processor 2602 (e.g., one or more centralprocessing units (CPUs), one or more graphics processing units (GPUs),one or more digital signal processors (DSPs), one or more applicationspecific integrated circuits (ASICs), one or more radio-frequencyintegrated circuits (RFICs), or any suitable combination thereof), amain memory 2604, and a static memory 2606, which are configured tocommunicate with each other via a bus 2608. The processor 2602 containssolid-state digital microcircuits (e.g., electronic, optical, or both)that are configurable, temporarily or permanently, by some or all of theinstructions 2624 such that the processor 2602 is configurable toperform any one or more of the methodologies described herein, in wholeor in part. For example, a set of one or more microcircuits of theprocessor 2602 may be configurable to execute one or more modules (e.g.,software modules) described herein. In some example embodiments, theprocessor 2602 is a multicore CPU (e.g., a dual-core CPU, a quad-coreCPU, an 8-core CPU, or a 128-core CPU) within which each of multiplecores behaves as a separate processor that is able to perform any one ormore of the methodologies discussed herein, in whole or in part.Although the beneficial effects described herein may be provided by themachine 2600 with at least the processor 2602, these same beneficialeffects may be provided by a different kind of machine that contains noprocessors (e.g., a purely mechanical system, a purely hydraulic system,or a hybrid mechanical-hydraulic system), if such a processor-lessmachine is configured to perform one or more of the methodologiesdescribed herein.

The machine 2600 may further include a graphics display 2610 (e.g., aplasma display panel (PDP), a light emitting diode (LED) display, aliquid crystal display (LCD), a projector, a cathode ray tube (CRT), orany other display capable of displaying graphics or video). The machine2600 may also include an alphanumeric input device 2612 (e.g., akeyboard or keypad), a pointer input device 2614 (e.g., a mouse, atouchpad, a touchscreen, a trackball, a joystick, a stylus, a motionsensor, an eye tracking device, a data glove, or other pointinginstrument), a data storage 2616, an audio generation device 2618 (e.g.,a sound card, an amplifier, a speaker, a headphone jack, or any suitablecombination thereof), and a network interface device 2620.

The data storage 2616 (e.g., a data storage device) includes themachine-readable medium 2622 (e.g., a tangible and non-transitorymachine-readable storage medium) on which are stored the instructions2624 embodying any one or more of the methodologies or functionsdescribed herein. The instructions 2624 may also reside, completely orat least partially, within the main memory 2604, within the staticmemory 2606, within the processor 2602 (e.g., within the processor'scache memory), or any suitable combination thereof, before or duringexecution thereof by the machine 2600. Accordingly, the main memory2604, the static memory 2606, and the processor 2602 may be consideredmachine-readable media (e.g., tangible and non-transitorymachine-readable media). The instructions 2624 may be transmitted orreceived over the network 2190 via the network interface device 2620.For example, the network interface device 2620 may communicate theinstructions 2624 using any one or more transfer protocols (e.g.,hypertext transfer protocol (HTTP)).

In some example embodiments, the machine 2600 may be a portablecomputing device (e.g., a smart phone, a tablet computer, or a wearabledevice), and may have one or more additional input components 2630(e.g., sensors or gauges). Examples of such input components 2630include an image input component (e.g., one or more cameras), an audioinput component (e.g., one or more microphones), a direction inputcomponent (e.g., a compass), a location input component (e.g., a globalpositioning system (GPS) receiver), an orientation component (e.g., agyroscope), a motion detection component (e.g., one or moreaccelerometers), an altitude detection component (e.g., an altimeter), atemperature input component (e.g., a thermometer), and a gas detectioncomponent (e.g., a gas sensor). Input data gathered by any one or moreof these input components may be accessible and available for use by anyof the modules described herein (e.g., with suitable privacynotifications and protections, such as opt-in consent or opt-outconsent, implemented in accordance with user preference, applicableregulations, or any suitable combination thereof).

As used herein, the term “memory” refers to a machine-readable mediumable to store data temporarily or permanently and may be taken toinclude, but not be limited to, random-access memory (RAM), read-onlymemory (ROM), buffer memory, flash memory, and cache memory. While themachine-readable medium 2622 is shown in an example embodiment to be asingle medium, the term “machine-readable medium” should be taken toinclude a single medium or multiple media (e.g., a centralized ordistributed database, or associated caches and servers) able to storeinstructions. The term “machine-readable medium” shall also be taken toinclude any medium, or combination of multiple media, that is capable ofcarrying (e.g., storing or communicating) the instructions 2624 forexecution by the machine 2600, such that the instructions 2624, whenexecuted by one or more processors of the machine 2600 (e.g., processor2602), cause the machine 2600 to perform any one or more of themethodologies described herein, in whole or in part. Accordingly, a“machine-readable medium” refers to a single storage apparatus ordevice, as well as cloud-based storage systems or storage networks thatinclude multiple storage apparatus or devices. The term“machine-readable medium” shall accordingly be taken to include, but notbe limited to, one or more tangible and non-transitory data repositories(e.g., data volumes) in the example form of a solid-state memory chip,an optical disc, a magnetic disc, or any suitable combination thereof.

A “non-transitory” machine-readable medium, as used herein, specificallyexcludes propagating signals per se. According to various exampleembodiments, the instructions 2624 for execution by the machine 2600 canbe communicated via a carrier medium (e.g., a machine-readable carriermedium). Examples of such a carrier medium include a non-transientcarrier medium (e.g., a non-transitory machine-readable storage medium,such as a solid-state memory that is physically movable from one placeto another place) and a transient carrier medium (e.g., a carrier waveor other propagating signal that communicates the instructions 2624).

Certain example embodiments are described herein as including modules.Modules may constitute software modules (e.g., code stored or otherwiseembodied in a machine-readable medium or in a transmission medium),hardware modules, or any suitable combination thereof. A “hardwaremodule” is a tangible (e.g., non-transitory) physical component (e.g., aset of one or more processors) capable of performing certain operationsand may be configured or arranged in a certain physical manner. Invarious example embodiments, one or more computer systems or one or morehardware modules thereof may be configured by software (e.g., anapplication or portion thereof) as a hardware module that operates toperform operations described herein for that module.

In some example embodiments, a hardware module may be implementedmechanically, electronically, hydraulically, or any suitable combinationthereof. For example, a hardware module may include dedicated circuitryor logic that is permanently configured to perform certain operations. Ahardware module may be or include a special-purpose processor, such as afield programmable gate array (FPGA) or an ASIC. A hardware module mayalso include programmable logic or circuitry that is temporarilyconfigured by software to perform certain operations. As an example, ahardware module may include software encompassed within a CPU or otherprogrammable processor. It will be appreciated that the decision toimplement a hardware module mechanically, hydraulically, in dedicatedand permanently configured circuitry, or in temporarily configuredcircuitry (e.g., configured by software) may be driven by cost and timeconsiderations.

Accordingly, the phrase “hardware module” should be understood toencompass a tangible entity that may be physically constructed,permanently configured (e.g., hardwired), or temporarily configured(e.g., programmed) to operate in a certain manner or to perform certainoperations described herein. Furthermore, as used herein, the phrase“hardware-implemented module” refers to a hardware module. Consideringexample embodiments in which hardware modules are temporarily configured(e.g., programmed), each of the hardware modules need not be configuredor instantiated at any one instance in time. For example, where ahardware module includes a CPU configured by software to become aspecial-purpose processor, the CPU may be configured as respectivelydifferent special-purpose processors (e.g., each included in a differenthardware module) at different times. Software (e.g., a software module)may accordingly configure one or more processors, for example, to becomeor otherwise constitute a particular hardware module at one instance oftime and to become or otherwise constitute a different hardware moduleat a different instance of time.

Hardware modules can provide information to, and receive informationfrom, other hardware modules. Accordingly, the described hardwaremodules may be regarded as being communicatively coupled. Where multiplehardware modules exist contemporaneously, communications may be achievedthrough signal transmission (e.g., over circuits and buses) between oramong two or more of the hardware modules. In embodiments in whichmultiple hardware modules are configured or instantiated at differenttimes, communications between such hardware modules may be achieved, forexample, through the storage and retrieval of information in memorystructures to which the multiple hardware modules have access. Forexample, one hardware module may perform an operation and store theoutput of that operation in a memory (e.g., a memory device) to which itis communicatively coupled. A further hardware module may then, at alater time, access the memory to retrieve and process the stored output.Hardware modules may also initiate communications with input or outputdevices, and can operate on a resource (e.g., a collection ofinformation from a computing resource).

The various operations of example methods described herein may beperformed, at least partially, by one or more processors that aretemporarily configured (e.g., by software) or permanently configured toperform the relevant operations. Whether temporarily or permanentlyconfigured, such processors may constitute processor-implemented modulesthat operate to perform one or more operations or functions describedherein. As used herein, “processor-implemented module” refers to ahardware module in which the hardware includes one or more processors.Accordingly, the operations described herein may be at least partiallyprocessor-implemented, hardware-implemented, or both, since a processoris an example of hardware, and at least some operations within any oneor more of the methods discussed herein may be performed by one or moreprocessor-implemented modules, hardware-implemented modules, or anysuitable combination thereof.

Moreover, such one or more processors may perform operations in a “cloudcomputing” environment or as a service (e.g., within a “software as aservice” (SaaS) implementation). For example, at least some operationswithin any one or more of the methods discussed herein may be performedby a group of computers (e.g., as examples of machines that includeprocessors), with these operations being accessible via a network (e.g.,the Internet) and via one or more appropriate interfaces (e.g., anapplication program interface (API)). The performance of certainoperations may be distributed among the one or more processors, whetherresiding only within a single machine or deployed across a number ofmachines. In some example embodiments, the one or more processors orhardware modules (e.g., processor-implemented modules) may be located ina single geographic location (e.g., within a home environment, an officeenvironment, or a server farm). In other example embodiments, the one ormore processors or hardware modules may be distributed across a numberof geographic locations.

Throughout this specification, plural instances may implementcomponents, operations, or structures described as a single instance.Although individual operations of one or more methods are illustratedand described as separate operations, one or more of the individualoperations may be performed concurrently, and nothing requires that theoperations be performed in the order illustrated. Structures and theirfunctionality presented as separate components and functions in exampleconfigurations may be implemented as a combined structure or componentwith combined functions. Similarly, structures and functionalitypresented as a single component may be implemented as separatecomponents and functions. These and other variations, modifications,additions, and improvements fall within the scope of the subject matterherein.

Some portions of the subject matter discussed herein may be presented interms of algorithms or symbolic representations of operations on datastored as bits or binary digital signals within a memory (e.g., acomputer memory or other machine memory). Such algorithms or symbolicrepresentations are examples of techniques used by those of ordinaryskill in the data processing arts to convey the substance of their workto others skilled in the art. As used herein, an “algorithm” is aself-consistent sequence of operations or similar processing leading toa desired result. In this context, algorithms and operations involvephysical manipulation of physical quantities. Typically, but notnecessarily, such quantities may take the form of electrical, magnetic,or optical signals capable of being stored, accessed, transferred,combined, compared, or otherwise manipulated by a machine. It isconvenient at times, principally for reasons of common usage, to referto such signals using words such as “data,” “content,” “bits,” “values,”“elements,” “symbols,” “characters,” “terms,” “numbers,” “numerals,” orthe like. These words, however, are merely convenient labels and are tobe associated with appropriate physical quantities.

Unless specifically stated otherwise, discussions herein using wordssuch as “accessing,” “processing,” “detecting,” “computing,”“calculating,” “determining,” “generating,” “presenting,” “displaying,”or the like refer to actions or processes performable by a machine(e.g., a computer) that manipulates or transforms data represented asphysical (e.g., electronic, magnetic, or optical) quantities within oneor more memories (e.g., volatile memory, non-volatile memory, or anysuitable combination thereof), registers, or other machine componentsthat receive, store, transmit, or display information. Furthermore,unless specifically stated otherwise, the terms “a” or “an” are hereinused, as is common in patent documents, to include one or more than oneinstance. Finally, as used herein, the conjunction “or” refers to anon-exclusive “or,” unless specifically stated otherwise.

The following enumerated embodiments describe various exampleembodiments of methods, machine-readable media, and systems (e.g.,machines, devices, or other apparatus) discussed herein.

A first embodiment provides a method comprising:

accessing, by one or more processors of a machine, a three-dimensional(3D) representation of a 3D object, the 3D representation defining a 3Dsurface of the 3D object;generating, by one or more processors of the machine, an atlas of colorinformation based on the 3D representation of the 3D object, thegenerating of the atlas including:for each cubic block that intersects the 3D surface among a plurality ofcubic blocks included in a bounding volume that encompasses the 3Dsurface, calculating an orthogonal projection of the cubic block onto acorresponding two-dimensional (2D) face of the bounding volume, theorthogonal projection defining a corresponding square block for thecubic block; andgrouping at least some of the square blocks that correspond to theorthogonally projected cubic blocks within a 2D image that specifies thecolor information generated based on the 3D representation of the 3Dobject; andproviding, by one or more processors of the machine, the generated atlasof the color information, the generated atlas including the groupedsquare blocks.

A second embodiment provides a method according to the first embodiment,wherein:

the accessing of the 3D representation of the 3D object includesaccessing at least one of a 3D point cloud that defines a set of 3Dpoints included in the 3D surface or a set of voxels that define the 3Dsurface.

A third embodiment provides a method according to the first embodimentor the second embodiment, wherein:

the generating of the atlas further includes:for each cubic block that intersects the 3D surface, determining thecorresponding 2D face of the bounding box by determining a correspondingdominant direction of the cubic block.

A fourth embodiment provides a method according to the third embodiment,wherein:

the generating of the atlas further includes:for a first cubic block that intersects the 3D surface among theplurality of cubic blocks, filling an empty area of a correspondingfirst square block by determining its color information based on afurther orthogonal projection of a second cubic block that is behind thefirst cubic block in the corresponding dominant direction.

A fifth embodiment provides a method according to the third embodimentor the fourth embodiment, wherein:

the determining of the corresponding dominant direction for a firstcubic block among the cubic blocks is based on a comparison of occludedareas among multiple orthogonal projections of the first cubic blockonto corresponding 2D faces of the bounding volume.

A sixth embodiment provides a method according to any of the firstthrough fifth embodiments, wherein:

in the generating of the atlas, the grouping of at least some of thesquare blocks is performed in accordance with a space partitioning tree.

A seventh embodiment provides a method according to any of the firstthrough sixth embodiments, wherein:

in the generating of the atlas, the grouping of at least some of thesquare blocks includes separately grouping square blocks defined byorthogonal projection onto each 2D face of the bounding volume.

An eighth embodiment provides a method according to any of the firstthrough seventh embodiments, wherein:

in the generating of the atlas, the grouping of at least some of thesquare blocks includes:segmenting a first region of the 2D image;determining a ratio of foreground information to background information;subdividing the first region of the 2D image based on a comparison ofthe determined ratio to a threshold ratio.

A ninth embodiment provides a method according to any of the firstthrough eighth embodiment, wherein:

the generating of the atlas further includes:calculating an average color of a first square block among the groupedsquare blocks; andrecoloring at least one empty pixel in the first square block with thecalculated average color.

A tenth embodiment provides a method according to any of the firstthrough ninth embodiments, further comprising:

performing video compression of the generated atlas of the colorinformation, the video compression generating a data stream thatincludes the grouped square blocks compressed in accordance with a videocodec; and whereinthe providing of the generated atlas of the color information providesthe generated data stream.

An eleventh embodiment provides a method according to any of the firstthrough tenth embodiments, wherein:

the providing of the generated atlas of the color information includesproviding the grouped square blocks along with indicators of theircorresponding dominant directions that indicate their correspondingfaces of the bounding volume.

A twelfth embodiment provides a method comprising:

accessing, by one or more processors of a machine, an atlas of colorinformation generated based on a three-dimensional (3D) representationof a 3D object, the atlas including grouped square blocks that representorthogonal projections of cubic blocks included in a bounding volumethat encompasses the 3D surface, the represented cubic blocksintersecting a 3D surface of the 3D object;by one or more processors of the machine, for each square block amongthe grouped square blocks, determining a corresponding two-dimensional(2D) face of the bounding volume by determining a corresponding dominantdirection of the corresponding cubic block that intersects acorresponding portion of the 3D surface; andby one or more processors of the machine, for each square block amongthe grouped square blocks, assigning corresponding color information toa 3D model of the corresponding portion of the 3D surface intersected bythe corresponding cubic block.

A thirteenth embodiment provides a method according to the twelfthembodiment, wherein:

the determining of the corresponding 2D face of the bounding volume forat least a first square block among the grouped square blocks includesaccessing a data stream that includes an indicator of the correspondingdominant direction of the corresponding cubic block.

A fourteenth embodiment provides a method according to the twelfthembodiment or the thirteenth embodiment, wherein:

the determining of the corresponding 2D face of the bounding volume forat least a first square block among the grouped square blocks is basedon a comparison of occluded areas among multiple orthogonal projectionsof the corresponding cubic block onto corresponding 2D faces of thebounding volume.

A fifteenth embodiment provides a method according to any of the twelfththrough fourteenth embodiments, wherein:

the accessing of the atlas of color information includes performingvideo decompression of the accessed atlas of the color information, thevideo decompression generating a 2D image that includes the groupedsquare blocks decompressed in accordance with a video codec.

A sixteenth embodiment provides a machine-readable medium (e.g., anon-transitory machine-readable storage medium) comprising instructionsthat, when executed by one or more processors of a machine, cause themachine to perform operations comprising:

accessing a three-dimensional (3D) representation of a 3D object, the 3Drepresentation defining a 3D surface of the 3D object;generating an atlas of color information based on the 3D representationof the 3D object, the generating of the atlas including:for each cubic block that intersects the 3D surface among a plurality ofcubic blocks included in a bounding volume that encompasses the 3Dsurface, calculating an orthogonal projection of the cubic block onto acorresponding two-dimensional (2D) face of the bounding volume, theorthogonal projection defining a corresponding square block for thecubic block; andgrouping at least some of the square blocks that correspond to theorthogonally projected cubic blocks within a 2D image that specifies thecolor information generated based on the 3D representation of the 3Dobject; andproviding the generated atlas of the color information, the generatedatlas including the grouped square blocks.

A seventeenth embodiment provides a system (e.g., a computer system)comprising:

one or more processors; anda memory storing instructions that, when executed by at least oneprocessor among the one or more processors, cause the system to performoperations comprising:accessing a three-dimensional (3D) representation of a 3D object, the 3Drepresentation defining a 3D surface of the 3D object;generating an atlas of color information based on the 3D representationof the 3D object, the generating of the atlas including:for each cubic block that intersects the 3D surface among a plurality ofcubic blocks included in a bounding volume that encompasses the 3Dsurface, calculating an orthogonal projection of the cubic block onto acorresponding two-dimensional (2D) face of the bounding volume, theorthogonal projection defining a corresponding square block for thecubic block; andgrouping at least some of the square blocks that correspond to theorthogonally projected cubic blocks within a 2D image that specifies thecolor information generated based on the 3D representation of the 3Dobject; andproviding the generated atlas of the color information, the generatedatlas including the grouped square blocks.

An eighteenth embodiment provides a machine-readable storage medium(e.g., a non-transitory machine-readable storage medium) comprisinginstructions that, when executed by one or more processors of a machine,cause the machine to perform operations comprising:

accessing an atlas of color information generated based on athree-dimensional (3D) representation of a 3D object, the atlasincluding grouped square blocks that represent orthogonal projections ofcubic blocks included in a bounding volume that encompasses the 3Dsurface, the represented cubic blocks intersecting a 3D surface of the3D object;for each square block among the grouped square blocks, determining acorresponding two-dimensional (2D) face of the bounding volume bydetermining a corresponding dominant direction of the correspondingcubic block that intersects a corresponding portion of the 3D surface;andfor each square block among the grouped square blocks, assigningcorresponding color information to a 3D model of the correspondingportion of the 3D surface intersected by the corresponding cubic block.

A nineteenth embodiment provides a system (e.g., a computer system)comprising:

one or more processors; anda memory storing instructions that, when executed by at least oneprocessor among the one or more processors, cause the system to performoperations comprising:accessing an atlas of color information generated based on athree-dimensional (3D) representation of a 3D object, the atlasincluding grouped square blocks that represent orthogonal projections ofcubic blocks included in a bounding volume that encompasses the 3Dsurface, the represented cubic blocks intersecting a 3D surface of the3D object;for each square block among the grouped square blocks, determining acorresponding two-dimensional (2D) face of the bounding volume bydetermining a corresponding dominant direction of the correspondingcubic block that intersects a corresponding portion of the 3D surface;andfor each square block among the grouped square blocks, assigningcorresponding color information to a 3D model of the correspondingportion of the 3D surface intersected by the corresponding cubic block.

A twentieth embodiment provides a system according to the nineteenthembodiment, wherein the operations further comprise:

for each cubic block that intersects the 3D surface of the 3D object,calculating an orthogonal projection of the cubic block onto acorresponding 2D face of the bounding volume that encompasses the 3Dsurface, the orthogonal projection defining the corresponding squareblock for the cubic block.

A twenty-first embodiment provides a carrier medium carryingmachine-readable instructions for controlling a machine to carry out themethod of any one of the previously described embodiments.

1. A method comprising: accessing, by one or more processors of amachine, a three-dimensional (3D) representation that defines a 3Dsurface of a 3D object; generating, by one or more processors of themachine, an atlas of color information based on the 3D representationthat defines the 3D surface of the 3D object, the generating of theatlas including: encompassing the 3D surface in a bounding volume thatincludes a plurality of cubic blocks that subdivides the 3D surface; foreach cubic block that intersects the 3D surface among the plurality ofcubic blocks included in the bounding volume, calculating acorresponding orthogonal projection of that cubic block onto acorresponding two-dimensional (2D) face of the bounding volume thatencompasses the 3D surface, the orthogonal projection of that cubicblock defining color information of a corresponding square block forthat cubic block; and grouping at least some of the square blocks thatcorrespond to the orthogonally projected cubic blocks within a 2D imagethat specifies the color information generated based on the 3Drepresentation that defines the 3D surface of the 3D object; andproviding, by one or more processors of the machine, the generated atlasof the color information, the generated atlas including the groupedsquare blocks.
 2. The method of claim 1, wherein: the accessing of the3D representation that defines the 3D surface of the 3D object includesaccessing at least one of a 3D point cloud that defines a set of 3Dpoints included in the 3D surface or a set of voxels that define the 3Dsurface.
 3. The method of claim 1, wherein: the generating of the atlasfurther includes: for each cubic block that intersects the 3D surface,determining the corresponding 2D face of the bounding volume bydetermining a corresponding dominant direction of that cubic block. 4.The method of claim 3, wherein: the generating of the atlas furtherincludes: for a first cubic block that intersects the 3D surface amongthe plurality of cubic blocks, filling an empty area of a correspondingfirst square block by determining its color information based on afurther orthogonal projection of a second cubic block that is behind thefirst cubic block in the corresponding dominant direction.
 5. The methodof claim 3, wherein: the determining of the corresponding dominantdirection for a first cubic block among the plurality of cubic blocks isbased on a comparison of occluded areas among multiple orthogonalprojections of the first cubic block onto corresponding 2D faces of thebounding volume.
 6. The method of claim 1, wherein: in the generating ofthe atlas, the grouping of at least some of the square blocks isperformed in accordance with a space partitioning tree.
 7. The method ofclaim 1, wherein: in the generating of the atlas, the grouping of atleast some of the square blocks includes separately grouping squareblocks defined by orthogonal projection onto each 2D face of thebounding volume.
 8. The method of claim 1, wherein: in the generating ofthe atlas, the grouping of at least some of the square blocks includes:segmenting a first region of the 2D image; determining a ratio offoreground information to background information in the first region;subdividing the first region of the 2D image based on a comparison ofthe determined ratio to a threshold ratio.
 9. The method of claim 1,wherein: the generating of the atlas further includes: calculating anaverage color of a first square block among the grouped square blocks;and recoloring at least one empty pixel in the first square block withthe calculated average color.
 10. The method of claim 1, furthercomprising: performing video compression of the generated atlas of thecolor information, the video compression generating a data stream thatincludes the grouped square blocks compressed in accordance with a videocodec; and wherein the providing of the generated atlas of the colorinformation provides the generated data stream.
 11. The method of claim1, wherein: the providing of the generated atlas of the colorinformation includes providing the grouped square blocks along withindicators of their corresponding dominant directions that indicatetheir corresponding 2D faces of the bounding volume.
 12. A methodcomprising: accessing, by one or more processors of a machine, an atlasof color information generated based on a three-dimensional (3D)representation that defines a 3D surface of a 3D object, the atlasincluding grouped square blocks whose color information is defined byorthogonal projections of cubic blocks included in a bounding volumethat encompasses the 3D surface, the represented cubic blocksintersecting and subdividing the 3D surface of the 3D object; by one ormore processors of the machine, for each square block among the groupedsquare blocks, determining a corresponding two-dimensional (2D) face ofthe bounding volume by determining a corresponding dominant direction ofthe corresponding cubic block that intersects a corresponding portion ofthe 3D surface; and by one or more processors of the machine, for eachsquare block among the grouped square blocks, assigning itscorresponding color information to a 3D model of the correspondingportion of the 3D surface intersected by the corresponding cubic block.13. The method of claim 12, wherein: the determining of thecorresponding 2D face of the bounding volume for at least a first squareblock among the grouped square blocks includes accessing a data streamthat includes an indicator of the corresponding dominant direction ofthe corresponding cubic block.
 14. The method of claim 12, wherein: thedetermining of the corresponding 2D face of the bounding volume for atleast a first square block among the grouped square blocks is based on acomparison of occluded areas among multiple orthogonal projections ofthe corresponding cubic block onto corresponding 2D faces of thebounding volume.
 15. The method of claim 12, wherein: the accessing ofthe atlas of color information includes performing video decompressionof the accessed atlas of the color information, the video decompressiongenerating a 2D image that includes the grouped square blocksdecompressed in accordance with a video codec.
 16. The method of claim12, further comprising: for each cubic block that intersects the 3Dsurface of the 3D object, calculating an orthogonal projection of thecubic block onto a corresponding 2D face of the bounding volume thatencompasses the 3D surface, the orthogonal projection defining the colorinformation of corresponding square block for the cubic block.
 17. Anon-transitory machine-readable storage medium comprising instructionsthat, when executed by one or more processors of a machine, cause themachine to perform operations comprising: accessing a three-dimensional(3D) representation that defines a 3D surface of a 3D object; generatingan atlas of color information based on the 3D representation thatdefines the 3D surface of the 3D object, the generating of the atlasincluding: encompassing the 3D surface in a bounding volume thatincludes a plurality of cubic blocks that subdivides the 3D surface; foreach cubic block that intersects the 3D surface among the plurality ofcubic blocks included in the bounding volume, calculating acorresponding orthogonal projection of that cubic block onto acorresponding two-dimensional (2D) face of the bounding volume thatencompasses the 3D surface, the orthogonal projection of that cubicblock defining color information of a corresponding square block forthat cubic block; and grouping at least some of the square blocks thatcorrespond to the orthogonally projected cubic blocks within a 2D imagethat specifies the color information generated based on the 3Drepresentation that defines the 3D surface of the 3D object; andproviding the generated atlas of the color information, the generatedatlas including the grouped square blocks.
 18. The non-transitorymachine-readable storage medium of claim 17, wherein: the generating ofthe atlas further includes: for each cubic block that intersects the 3Dsurface, determining the corresponding 2D face of the bounding volume bydetermining a corresponding dominant direction of that cubic block. 19.A system comprising: one or more processors; and a memory storinginstructions that, when executed by at least one processor among the oneor more processors, cause the system to perform operations comprising:accessing a three-dimensional (3D) representation that defines a 3Dsurface of a 3D object; generating an atlas of color information basedon the 3D representation that defines the 3D surface of the 3D object,the generating of the atlas including: encompassing the 3D surface in abounding volume that includes a plurality of cubic blocks thatsubdivides the 3D surface; for each cubic block that intersects the 3Dsurface among the plurality of cubic blocks included in the boundingvolume, calculating a corresponding orthogonal projection of that cubicblock onto a corresponding two-dimensional (2D) face of the boundingvolume that encompasses the 3D surface, the orthogonal projection ofthat cubic block defining color information of a corresponding squareblock for that cubic block; and grouping at least some of the squareblocks that correspond to the orthogonally projected cubic blocks withina 2D image that specifies the color information generated based on the3D representation that defines the 3D surface of the 3D object; andproviding the generated atlas of the color information, the generatedatlas including the grouped square blocks.
 20. The system of claim 19,wherein: in the generating of the atlas, the grouping of at least someof the square blocks includes: segmenting a first region of the 2Dimage; determining a ratio of foreground information to backgroundinformation in the first region; subdividing the first region of the 2Dimage based on a comparison of the determined ratio to a thresholdratio.